Skip to content

chore(docs): upgrade Docosaurus to 3.9.2 with Rspack and SWC optimizations#68600

Merged
letiescanciano merged 5 commits intomasterfrom
leti/upgrade-docusaurus
Oct 24, 2025
Merged

chore(docs): upgrade Docosaurus to 3.9.2 with Rspack and SWC optimizations#68600
letiescanciano merged 5 commits intomasterfrom
leti/upgrade-docusaurus

Conversation

@letiescanciano
Copy link
Contributor

Summary

Upgrade Docosaurus from 3.7.0 to 3.9.2 with major performance improvements:

  • Rspack bundler: 5-10x faster than webpack (written in Rust)
  • Rspack persistent cache: ~80% faster on subsequent builds
  • SWC minifier: 20-50x faster than Terser for JavaScript minification
  • TypeScript config: Full IntelliSense support for all config options
  • Memory optimization: Increased Node.js heap to 3GB to prevent swapping

Changes

  • Upgrade @docusaurus/core and all related packages to 3.9.2
  • Upgrade @signalwire/docosaurus-plugin-llms-txt to 1.2.2
  • Enable experimental_faster flags (rspackBundler, rspackPersistentCache,
    swcJsMinimizer, swcJsLoader)
  • Convert docosaurus.config.js to TypeScript (docosaurus.config.ts)
  • Add tsconfig.json for TypeScript support

Expected Improvements

Build time on Vercel (4 cores, 8GB):

  • Before: ~28-30 minutes
  • After: ~10-12 minutes first build, 5-7 minutes subsequent builds
  • Expected improvement: 70-80% faster builds

Testing

  • Build tested locally with new Docosaurus 3.9.2 configuration
  • TypeScript configuration verified with proper types

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

@github-actions
Copy link
Contributor

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Helpful Resources

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • /format-fix - Fixes most formatting issues.
  • /bump-version - Bumps connector versions.
    • You can specify a custom changelog by passing changelog. Example: /bump-version changelog="My cool update"
    • Leaving the changelog arg blank will auto-populate the changelog from the PR title.
  • /run-cat-tests - Runs legacy CAT tests (Connector Acceptance Tests)
  • /build-connector-images - Builds and publishes a pre-release docker image for the modified connector(s).
  • JVM connectors:
    • /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
      Example: /update-connector-cdk-version connector=destination-bigquery
    • /bump-bulk-cdk-version type=patch changelog='foo' - Bump the Bulk CDK's version. type can be major/minor/patch.
  • Python connectors:
    • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
    • /poe source example lock - Alias for /poe connector source-example lock.
    • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
    • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.

📝 Edit this welcome message.

return {
type: "category",
label: contentTitle,
key: `${prefix}${filename}-category`,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need keys for sidebar items to generate unique translations keys

@letiescanciano letiescanciano force-pushed the leti/upgrade-docusaurus branch from 7549f96 to e7657f0 Compare October 23, 2025 09:56
@github-actions
Copy link
Contributor

github-actions bot commented Oct 23, 2025

Deploy preview for airbyte-docs ready!

✅ Preview
https://airbyte-docs-4szvidznc-airbyte-growth.vercel.app

Built with commit 30bb483.
This pull request is being automatically deployed with vercel-action

@letiescanciano letiescanciano marked this pull request as ready for review October 23, 2025 17:06
Copy link
Contributor

@ian-at-airbyte Ian Alton (ian-at-airbyte) left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified several things.

  • Build times are around 5 minutes\ (with 8 cores / 16 GB)
  • Mermaid diagrams display.
  • API docs display.
  • Code samples imported from GitHub display.
  • Connectors and migration guides display for AB and Enterprise connectors.
  • MarkDown includes between files work correctly.
  • Search works (and looks a bit nicer thanks to the new version of Algolia).
  • Click around the site and spot checked things.

Could not verify:

  • Chameleon survey doesn't open after clicking "no" on the feedback question. That might just be because it only runs on docs.airbyte.com though, probably not an issue.
  • I did notice some API pages are "floating" with no sidebar (example), but that seems unrelated to the work here and might be an API-specific problem.

I can find no fault with anything here.

Comment on lines +33 to +38
experimental_faster: {
rspackBundler: true,
rspackPersistentCache: true,
swcJsMinimizer: true,
swcJsLoader: true,
},

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't experimental_faster: true, enable all the speed improvements already?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it did but it turns out it only selects a few, according to Claude. I need to still find the docs, but I also thought it was more descriptive outlining what we wanted to enable

Copy link
Contributor Author

@letiescanciano letiescanciano Oct 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only thing I've found in the docs it's that it should enable all of them. facebook/docusaurus#10931

but if I use experimental_faster:true I get this error:

[ERROR] Site reload failure                                             
 Error: Docusaurus config `future.experimental_faster.ssgWorkerThreads` 
requires the future flag `future.v4.removeLegacyPostBuildHeadAttribute` to be turned on.

so I'll add the ones I was missing, and don't use ssgWorkerThreads until we want to enable v4 flags

@letiescanciano letiescanciano force-pushed the leti/upgrade-docusaurus branch from ceff83f to 9789ceb Compare October 24, 2025 04:22
Reorganize connector registry management for better separation of concerns:

Build-time improvements:
- Create fetch-registry.js: Centralized utility for fetching and caching registry
  - Handles cache check, network fetch fallback, and minimal data extraction
  - Reusable by all build scripts without duplication
  - Reduced console logging for cleaner build output
- Rename prepare-sidebar-data.js to prepare-registry-cache.js
  - More accurately reflects its purpose as a registry cache preparer
  - Updated package.json scripts to use new name
- Consolidate constants into src/scripts/constants.js
  - Centralized registry paths, URLs, and doc directory constants
  - Used by all scripts: fetch-registry, prepare-registry-cache, cleanup-cache
- Simplify connector_registry.js to original approach (network fetch only)
  - No complex caching logic (handled by fetch-registry now)
  - Maintains original compatibility with remark plugins

Client-side improvements:
- Create clientRegistryUtils.js: Browser-safe registry lookup
  - Uses fetch to retrieve registry at runtime
  - Separated from build-time utilities to prevent webpack bundling issues
- Update RequestERD and EntityRelationshipDiagram components
  - Use clientRegistryUtils instead of build-time registry
  - Components now self-contained and independent of build process

Webpack fixes:
- No longer attempt to bundle Node.js code (fs, path modules)
- Remark plugins now use fetch-registry (async, build-time only)
- Client components use clientRegistryUtils (browser-safe)

Benefits:
✓ Clean separation: Build-time vs Runtime vs Client-side code
✓ Single responsibility: Each script has one clear purpose
✓ DRY principle: No duplicate constants or logic
✓ Reduced logging: Cleaner build output
✓ Better maintainability: Easier to understand data flow
@letiescanciano letiescanciano merged commit 72b6301 into master Oct 24, 2025
32 checks passed
@letiescanciano letiescanciano deleted the leti/upgrade-docusaurus branch October 24, 2025 05:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants